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'Phis  report  includes  a  description  of  the  progress  and  accomplishments  achieved 
on  our  research  program  for  the  period  June  15,  1988  to  November  14,  1900  .  Prelim¬ 
inary  reports  and  publications  of  the  research  results  are  listed  in  section  6  .  Papers 
presented  at  conferences  and  seminars  are  listed  in  Section  7. 

Our  proposal  is  in  the  general  area  of  Nonlinear  Optimization  and  the  research 
issues  focus  on  duality  methods  for  infinite  dimensional  optimization  problems  and 
applications.  The  proposal  covered  the  following  areas: 

I  Duality  methods  in  convex  optimization 

2.  Applications  of  generalized  entropy 

It,  Semi-inlinile  programming 


'1.  Nonsmoolh  Optimization 
5,  HIocIiunI  ic  progi'fiimuing 

Itcsults  accomplished  since  this  proposal  has  been  submitted  in  October  1987,  in  the 
areas  covered  by  the  current  grant  are  summarized  below. 
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1.  Duality  methods  in  convex  optimization 

The  results  of  the  research  in  I  his  area  art-  described  in  (lie  following  papers  jlj, 
[2],  (7).  In  |lj  we  develop  a  general  duality  theory  for  minimizing  a  convex  objective 
fund  ion  <  >ver  an  infinil  e  dimensional  c<  m  vex  o  >ne  subject  to  a  finite  number  of  equal  it  v 
constraints,  for  a  wide  class  of  objective  functions  we  are  able  to  produce  an  explicit 
dual  problem.  This  dual  problem  is  typically  finite  dimensional  and  smooth.  It  can 
often  be  solved  effect ivelv  and  the  infinite  dimensional  primal  solution  can  be  simply 
recovered.  We  apply  our  results  to  the  problem  of  estimating  a  spectral  density  given 
finitely  many  correlations.  The  spectral  estimation  problem  is  formulated  as  a  convex 
programming  problem  on  l.r  spaces  with  equality  and  non-negativity  constraints  We 
demonstrate  how  classical  spectral  estimation  results  can  be  derived  in  this  framework 
and  how  significant  general  results  may  be  obtained  on  using  generalized  entropic 
fund  ii  >us. 

Knit  ropy  optimization  problems  which  seek  to  maximize  the  Shannon  entropy  func¬ 
tional  or  to  minimize  t ho  Rollback- Liebler  discrimination  information  measure  be¬ 
tween  two  probability  measures  over  linear  equality  and  inequality  constraints  arise 
in  various  liclds  of  applications.  Several  interesting  problems  in  engineering  and  so¬ 
cial  sciences  include'  also  in  the  constraint  set  quadratic  inequality  constraints.  In  [ 2 ] 
we  derive  duality  results  for  these  problems  Our  approach  is  a  di red  application  of 
bagrangian  duality  and  is  extended  to  a  more  general  class  of  entropy  optimization 
problems  with  composite  convex  constraints  The  results  allow  us  to  establish  a  sim¬ 
ple  and  computationally  tractable  bagrangian  dual  problem  with  only  nonnegnli vif v 
const  rain!  s. 

Constraints  qualifications  in  infinite  dimensional  programming  problems  are  of 
fundamental  importance  to  derive  duality  results.  Many  seemingly  unrelated  (CQ) 
have  been  recently  proposed  in  the  literature.  In  a  joint  venture  with  I)r.  M.  S. 
Gowda,  we  have  investigated  the  relationships  between  old  and  new  (CQ)  in  an  effort 
to  provide  a  unified  framework  to  derive  these  (CQ).  This  work  is  summarized  in  (7). 
Using  Robinson’s  refinement  of  the  duality  result  of  Rockafeller,  we  demonstrate  that 
the  constraint  qualification  proposed  bv  Rockafeller  provides  a  systematic  mechanism 
for  comparing  many  constraints  qualifications  ns  well  as  to  establish  new  results  in 
different  topological  environments.  Our  results  and  examples  demonstrate  that  these 
(CQ)  seem  to  In  the  sharpest  v<  rifiahle  conditions  I  hat  can  be  used  in  deriving  duality 
results  for  the  class  of  infinite  dimensional  optimization  problems  we  have  considered 
in  this  proposal. 


2.  Applications  of  generalized  entropy 

Tin'  results  of  our  research  m  this  area  are  summarized  in  [3],  (  I).  In  [3]  we  provide 
a  systematic  survev  of  application  of  generalized  entropy  functional  to  a  diversity  of 
problems.  These  applications  include,  stochastic  programming.  Information  theory, 
game  theory,  spectral  estimation,  rharari  erizat  ion  of  means  and  characterization  of 
probability  measures. 

In  jl]  we  investigate  the  applicability  of  generalized  entropy  in  the  context  of 
economics  under  uncertainty.  Pualitv  and  extremal  principles  for  certainty  equivalent 
(  in  the  sense  of  expected  utility)  and  information  measures  are  derived.  We  establish 
fundamental  relations  between  the  notion  of  divergence  DIV  (generalized  entropy)  of 
Csiszar  and  two  objects  used  in  the  study  of  uncertainty,  namely  E  (expectation), 
(T'l  (certainly  equivalent).  Each  member  of  the  triple  (E,  (IE,  DTV)  is  the  optimal 
value  of  an  extremum  problem  involving  the  sum  or  difference  of  the  other  two.  These 
relations  provide  a  duality  framework  for  economics  under  uncertainty  and  entropy, 
giving  new  concrete  economic  interpretations. 

3.  Semi-infinite  linear  programs 

Most  of  I  he  existing  numerical  methods  for  solving  semi-infinite  problems  are  based 
on  discretization  and  approximation  techniques.  In  our  proposal  we  have  suggested 
a  new  idea  which  does  not  attempt  to  destroy  the  continuum  of  the  constraints  by 
discretization.  Our  approach  is  essentially  to  look  at  a  linear  program  in  infinite 
dimension  with  a  perturbed  objective  function.  Ibis  allow  us  to  derive  a  sequence  of 
dual  problems  in  finite  dimension  having  the  property  of  penalizing  the  constraints  of 
a  semi-infinite  dual  linear  problem.  This  open  the  possibility  of  solving  semi-infinite 
linear  programs  via  unconst  rained  optimizat  ion  techniques.  A  preliminary  analysis  is 
summarized  in  [11;. 

4.  Nonsmootli  Optimization 

In  the  nature  of  research,  new  results  in .t  anticipated  when  writing  the  proposal,  have 
lead  to  investigate  fpiiorm  minimization  problems  and  related  non  smooth  problems 
and  their  corresponding  duals.  Results  in  this  area  are  given  in  [5],  [8 1 ,  [l()|.  In  [ F> j . 
we  proved  that  the  solution  of  a  least  squares  problem  of  an  overdetermined  system 
of  linear  equations  is  in  the  '-onvex  hull  .s’  of  points,  each  of  which  is  a  solution  of  a 
square  subsystem  of  the  whole  system  I  lie  result  remain  I  rue  fin  a  weighted  least 
squares  solution,  with  a  positive  definite  weighting  matrix  and  for  fr-uorm  minimiza¬ 
tion  problems.  1  p  oo.  The  algorithmic  imnlienl  ••>•!«  of  these  '■'•-.nits  is  ■.;!  {•>  lie 
explored  in  the  context  o|  /,,-norm  problems  and  of  linear  programming,  where  the 
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soliit.ii >11  <>['  a  least  squares  problem  is  in  the  heart  of  Karmarkar’s  method.  In  [8], 
we  develop  a  fairly  general  framework  for  smoothing  nondiflerent  iable  optimizat  ion 
problems.  The  methodology  uses  the  concept  of  recession  function  to  construct,  an 
approximate  perturbed  problem.  A  priori  error  bounds  and  corresponding  perturbed 
duals  are  obtained.  Another  direction  of  research  in  this  area  was  to  apply  our  results 
to  some  problems  in  applied  mechanics.  This  has  been  accomplished  in  a  joint  work 
with  Drs.  A.  Ben-Tal  and  W.  Vang.  Our  results  are  summarized  in  [fi].  We  consider 
there.  I  lie  solution  of  a  wide  class  of  limit  analysis  problems  (asymptotic  behavior  of 
elastic-plastic  materials)  that  can  be  formulated  as  a  pair  of  primal-dual  optimization 
problems,  the  later  being  typically  a  /riinrm  optimization  problem.  To  overcome  the 
non  differentiability  of  the  objective  function,  we  replace  the  original  problem  by  an 
approximate  smoothed  one,  which  is  controlled  by  a  smoothing  parameter.  A  new 
Combined  Smoothing  Successive  Approximation  (CSSA)  method  is  then  developed 
for  solving  this  problem.  The  convergence  of  the  CSSA  algorithm  is  proved  and  some 
apriori  error  estimates  are  obtained.  The  performance  of  the  above  algorithm  has 
some  remarkable  characteristics;  in  each  of  the  numerical  experiments  a  practically 
acceptable  solution  was  obtained  after  only  few  iterations  even  from  very  bad  start¬ 
ing  points,  and  for  a  wide  range  of  the  smoothing  parameter.  In  particular,  the  first 
iteration  always  produces  a  solution  vector  (|ui!e  close  to  the  final  solution  even  with 
randomly  generated  initial  solution.  The  subsequent  iterations  are  less  dramatic  and 
arc  observed  to  he  progressively  slower.  extrapolation  ran  Ilian  be  used  to  accelerate 
convergence.  To  illustrate  our  results  and  give  numerical  evidence,  two  examples  of 
truss  and  plate  structures  are  solved. 

Recently  we  studied  and  develop  a  general  framework  for  rogularizalon  and  ap¬ 
proximation  of  nonsmooth  optimization  problems  via  proximal-like  methods  and  aug¬ 
mented  Lagrangians.  Initial  results  are  described  in  |l3j,  [14]. 

5.  Stochastic  Programming; 

In  [6j,  with  .1.  Bilge,  new  upper  bounds  are  given  for  the  expected  value  of  a  convex 
function.  The  bounds  employ  subgradient  information  and  the  conjugate  function. 
Tn  contrast  to  most  other  bounds,  explicit  moment  information  is  not  needed.  These 
results  are  especially  useful  when  (1)  the  integral  is  difficult  to  evaluate;  (2)  the 
moments  are  unknown  or  do  not  exist ;  (.‘II  the  integrand  is  unbounded  over  the  support 
of  the  random  vector.  Examples  illustrate  the  results.  In  particular,  we  consider 
integrals  with  logarithmic  penalties  and  practical  cases  with  unbounded  moments. 
We  also  show  how  the  hound  mav  he  particularly  useful  for  integrating  piecewise 
linear  convex  functions  in  multiple  dimensions.  such  as  the  optimal  value  function 
of  a  stochastic  linear  program.  In  ;9|.  the  portfolio  selection  problem  with  one  safe 
and  a  risk \  assets  is  anaiv/ed  va  a  n*  decision  ilieoietic  criterion,  the  decourse 
Certain!  v  equivalent  (RCK),  introduced  hv  the  authors  in  19Kb.  Fundamental  results 


in  portfolio  theory,  previously  studied  under  the  expected  Utility  criterion  (Fdl),  such 
hs  separation  t  heorems,  cmnparat  ive  stal  ic  analysis,  and  t  hresliold  values  for  inclusion 
or  exclusion  of  risky  assets  in  the  optimal  portfolio,  are  obtained.  In  contrast  to  the 
UU  model,  our  results  lor  the  RCK  maximizing  investor  do  not  impo".'  restriction-' 
on  either  the  utility  function  or  the  underlying  probability  laws. 

A  list  of  articles,  written  or  revised  and  in  preparation  under  the  grant  follows. 
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G.  Grant.  Supported  Manuscripts 

Published  and  accepted  for  publication 

ilj  Spectral  Est imation  via  Convex  Programming”,  Essays  in  Honor  of  Abraham 
Charlies.  Eds.  A.  Hen-lsrael  el  at.  (will,  A  Ben-Tal  and  .1 .  M.  Horwein  ).  In 
Press. 

12)  "Quadrat  icallv  Constrained  Pail  ropy  Functionals  and  Extensions’’,  SIAM  J.  of 
Applied  Mathematics.  49,  1989,  pp.  1895-  1850. 

[3j  ’’  On  'F- Divergence  and  its  Applications”,  Essays  m  Honor  of  Abraham  Char  nos, 
Eds.  A.  Ben-lsrael  et  al.  In  Press. 

1*1 1  ’’Certainty  Equivalents  and  Information  Measures:  Duality  and  Extremal  Prin¬ 
ciples”,  .7.  of  Mathematical  Analysis  and  Applications  (with  A.  Ben-Tal  and  A. 
lien -Israel).  In  Press. 

la)  "(.leometric  Properties  of  the  beast  Square  Solution  of  Linear  Equations”,  Lin¬ 
ear  Algebra  and  Applications.!  39.  1990,  pp. 105-170  (with  A.  Ben-Tal). 

j(ij  ’Upper  bounds  on  the  expected  value  of  a  convex  function  using  gradient  and 
conjugate  function  informal  ion” .  Mathematics  of  Operations  Research,  14,  1989, 
pp.  715-759.  (with  .1.  Birge) 

1 7 1  ”  A  Comparison  of  Constraint  Qualifications  in  Infinite  Dimensional  Convex 
Programming  ”,  SIAM  ./.  of  Control  and  Optimization ,  28,  1990,  pp.  925-935. 
(with  M .  S.  (  e >wda). 

|K]  ”  A  smoothing  technique  for  nondifferent iahle  optimization  problems”,  Op¬ 
timization,  Lecture  Notes  in  Mathrmalics  rol.  Uf05,  Ed.  S.  Dolocki,  1989, 
pp.(with  A.  Ben-Tal).  1-12. 

1 9 ]  ’’Portfolio  Theory  for  the  recourse  certainty  equivalent  maximizing  investor”, 
Annals  of  Operations  Research ,  (with  A.  Ben-Tal).  In  Press. 

flOj  ”  A  Combined  Smoothing  Successive  Approximation  Method  for  Limit  Analysis 
Problems”.  .7.  of  Applied  Math,  and  Optimization,  (with  A.  Ben-Tal  and  \V.  11. 
Yang).  In  Press. 

)ll|  "Null  Linear  Perturhat  ions  in  linear  semi  inlinii  e  programming  problems” ,  //.’/■,’ 
(.'DC  proceedings  on  Decision  and  Control  In  Press. 

Submitted  for  publication 

[  1 2 j  ”A  primal-dual  iterative  algorithm  for  a  maximum  likelihood  estimation  prob¬ 
lem”,  (with  A.  N.  lusetn). 
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!  I.'ij  ’’Kntropic  proximal  mappings  wit  h  applirat  ions  lo  nonlinear  programming" . 

[  1-1]  "  ('onvergence  analysis  of  a  proximal-like  minimization  algorithm  using  Bregman 
fuuct  ions" . 
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7.  Conferences,  Meetings,  Seminars 

1 .  A  MS-SI  A  M  Summer  Seminar  on  Com  pul  at  ion  al  Sol ul  ion  <  >f  Nonlinear  S  vst  cms 
of  F.ijuat  ions,  .1  uly  I S  29.  1 988  .  ( \  >1<  undo  Si  ate  University,  Ft .  (loll  ins  Colorado. 
(In vii ed  pari icipant .) 

2.  IU/TCOH,  Rutgers  Center  I  or  Operations  Hesearrli,  Mardi  21,  1989,  New 
Brunswick,  N.  .1.,  Operations  Research  Seminar.  ”  NotulifT'ercnl iahle  Optimiza¬ 
tion  via  Smoothing  and  Approximaf ion" .  (Invited  lecture) 

2.  SLAM  Conference  on  Optimization.  April  2-5,  1989,  Boston,  Mass. 

’’Duality  methods  for  Finite  and  Infinite  Dimensional  Optimization  Problems”. 
(Contributed  paper.) 

1.  Mathematical  Programming  Workshop,  April  28,  1989,  The  Johns  Hopkins  Uni¬ 
versity,  Baltimore.  Ml).  ’’Nonsmooth  Optimization”.  (Invited  lecture) 

a.  Fdeven  Symposium  on  Mathematical  Programming  with  Data  Perturbations, 
May  25-20,  1989,  Washington.  IF  0  ”  Smoothing  and  Approximation  Methods 
in  Nondifferent iahle  0|it imizat ion"  (Invited  talk) 

fi.  Operation  Research  Soviet v  of  Israel  Mi'eting.  May  1990.  ’’Entropy-like  proxi¬ 
mal  mappings  and  applications"  (Invited  talk). 


